The Internet provides access to a wide variety of resources, for example, video files, image files, audio files, or Web pages, including content for particular subjects, reference articles, or news articles. A typical user can select a particular web resource she is wishing to access using a browser application executed on an electronic device, be it a desktop computer, a laptop computer, a tablet or a smartphone. There is a number of commercially available browsers to execute such a function, GOOGLE CHROME browser, INTERNET EXPLORE browser, YANDEX browser and the like. The user can type in a Universal Resource Locator (URL) of the web resource she is wishing to access or, alternatively, the user can select (click or otherwise actuate) a hyperlink to the URL of the web resource she is desirous of accessing.
The above approach works when the user is aware apriori of the web resource she is wishing to access. For example, if the user knows apriori that she wishes to access a web site of the Royal Bank of Canada, she can type into a browser of her choice, the URL associated with the Royal Bank of Canada, which can be www.rbc.com.
However, a given user may not know a specific web resource that the user wants to access, but rather know a type of information the user is looking for. In those circumstances, the user may use a so-called search engine to locate one or more web resources that provide information that the user is interested in. To that extent, the user can submit a “search query” and the search engine returns a ranked list of search results that are responsive to the search query in a form of a Search Engine Results Page (or SERP for short).
With reference to FIG. 9, there is provided a screen shot 9100 of a typical SERP provided by a prior art search engine, in this case, the search engine being implemented as YANDEX search engine, provided by Yandex LLC of 16, Leo Tolstoy St., Moscow 119021, Russia. For illustration purposes, the search engine that generated the screen shot 9100 is executed on a desktop computer.
Within the illustrated scenario, the user has typed into a search query interface 9102 of the search engine a search query “Ecuador”, presumably interested in information about Ecuador. As will be appreciated, the search query “Ecuador” does not have a clear search intent, as the user typing in such the query may be interested in getting information about the country Ecuador, about the song “Ecuador” by Sash!, getting news about Ecuador or getting pictures of Ecuadorian landscapes.
The search query entered into the search query interface is transmitted to a search engine server (not depicted) and the search engine server executes a search and returns data for generating a SERP 9104. The SERP 9104 is configured to convey to the user one or more search results. These search results, as well as their presentation, will vary, but generally and as an example only will include: a first search result 9106, a second search result 9108, a third search result 9110 and a plurality of additional search results 9112. Some of these search results can be considered “web search results” and some of these search results can be considered to be “vertical search results”. The web search results (such as the first search result 9106 and the third search result 9110, for example) are search results returned by a web search module of the search engine and are generally web resources available on the Internet (in these case, these are Russian article about Ecuador on WIKIPEDEA and Lonely Planet article about Ecuador, respectively). The vertical search results (such as the second search result 9108, for example) are search results returned by one or more of the vertical search modules of the search engine (in this case, the second search result 9108 is implemented as a “widget” presenting results of the video vertical—i.e. one or more videos that are responsive to the search query “Ecuador”).
Optionally, the SERP 9104 may also include an object card 9114. The object card 9114 is typically presented when the search engine determines that the search query is associated with an “object”, the object typically includes either a person (an actor, a singer, a politician or the like), a point of interest (such as a bridge, a museum, a city hall, a train station and the like) or any other entity (such as a movie, a play and the like).
The SERP 9104 may also include vertical domain actuator 9120, which is configured to allow the user to select (and change) a particular search domain—in the illustrated example, the user can cause the SERP 9104 to switch from the current “web” view of search results, to one or more vertical domains, including: “maps”, “images”, “news”, “videos” and the like. The number and exact types of the vertical domains can differ, but vertical domains allow the user to switch to a particular type of search results. For example, if the user was interested in images of Ecuador, the user could switch to the “images” vertical, which would cause the SERP 9104 to change and to present to the user search results from the “images” vertical, the search result being images that are responsive the search query “Ecuador”.
One will easily appreciate that the search results shown as part of the plurality of additional search results 9112 is not an entirety of all search results that the search engine has generated in response to the search query. On the contrary, the plurality of additional search results 9112 includes many more search results that are not visible within the screen shot 9100, due to the limitations of the real estate of a display of the electronic device. Furthermore, search engines typically “split” search results in several screens and to that end a scroll actuator 9116 is provided to switch to the “next” portion of the SERP 9104. The scroll actuator 9116 can be an arrow, a numeric indicator of screens within the SERP 9104 or the like.
One of the technical challenges for the search engine server, is to select and rank search results to generate the SERP 9104 that is “time effective” for the user. What this means is that search engines strive to put the most relevant search results (i.e. the search results that are more likely to satisfy the user's search intent” towards the “top” of the SERP 9104. In other words, the search results presented on higher positions of the SERP 9104 (i.e. first n-number of search results shown on the first page of the SERP 9104) should be able to satisfy the user's search intent. There is a general belief in the industry that if the user has to “scroll” through the search results to the second, third, etc. pages of the SERP 9104, the “quality” of the SERP 9104 is deemed to be lower than desired.
Search engines employ various techniques and algorithms for ranking search results. Typically, a machine learning algorithm is used for ranking the search results into the SERP 9104. Various techniques are available for ranking search results. Just as an example, some of the known techniques for ranking search results by relevancy are based on some or all of: (i) how popular a given search query or a response thereto is in other prior searches (web or vertical); (ii) how many results have been returned by either the vertical or web search modules; (iii) whether the search query contains any determinative terms (such as “images”, “movies”, “weather” or the like), (iv) how often a particular search query is typically used with determinative terms by other users; and (v) how often other uses performing a similar search have selected a particular resource or a particular vertical search results when results were presented using the SERP 9104.
One of the parameters used by prior ranking algorithms, especially for ranking vertical search results relative to the web search results, is a so-called “usefulness parameter”. A typical prior art system can rank search results based on a Click Through Rate (CTR) rate analysis of a first search result (a higher ranked search result, which is typically a vertical search result), typically referred to as a “win” and a second search result (following immediately after the first search result, which is typically but not necessarily a web search result), typically referred to as a “loss”. The function that is typically used is “S(ƒ, iw)−win−loss”, where ƒ is ranking features, iw is a parameter that is indicative of the position of the given search result. The ranking features can include one or more of: word occurrence probability, behavioural patterns, personalized parameters respectively associated with the first search result and the second search result. The iw parameter can include the rank of the first search result and the second search result, its associated intent weight (i.e. a parameter indicative of potential user need in search results of a particular category—images, video, maps, news, etc).
Within the prior art solutions, a machine learning algorithm is trained to predict the usefulness parameter. The machine learning algorithm is trained using: (i) as input parameters, a rank of the search result and the associated ranking features, the associated ranking features having been determined based on an analysis of prior (i.e. historic) search sessions performed by other users; (ii) as labelled answers—the CTR value for the given SERP position.
As part of the machine learning algorithm training, the machine learning algorithm established a relationship between (i) the value of the usefulness parameter function “S(ƒ, iw)−win−loss” on one hand and (ii) ranking features (including iw feature) on the other hand. The usefulness parameter formula is then used to select a particular position for a given vertical search result within the SERP, the particular position being selected such that to maximize the usefulness parameter for the given vertical search result.
U.S. Pat. No. 8,706,725 teaches methods for re-ranking documents based on user-specific features. Search results are received from a non-contextual ranking system such that the search results are not specific toward a particular user, such as the user who submitted the search query. Contextual signals are received and provide user-specific features that are used to re-rank documents so that the most important and relevant documents are listed at the top of the list of search results. Each of the user-specific features are evaluated and compared to determine a new position of each document. A set of contextual search results is then generated based on the new positions.
U.S. Pat. No. 8,650,173 discloses technologies for placing search results on a search engine results page (SERP). A query may be received. The query may be transmitted to a plurality of search result providers. A first set of search results and a second set of search results may be received from the search result providers. Intent features may be extracted from the first set of search results. User intent of the second set of search results may be inferred based on the extracted intent features. The first set of search results and the second set of search results may be ranked based on the inferred user intent. The SERP may be rendered according to the ranked first set and second set of search results.
U.S. Pat. No. 7,698,331 teaches a system for generating a search result list in response to a search request from a searcher using a computer network. A first database is maintained that includes a first plurality of search listings. A second database is maintained that includes documents having general web content. A search request is received from the searcher. A first set of search listings is identified from the first database having documents generating a match with the search request and a second set of search listings is identified from the second database having documents generating a match with the search request. A confidence score is determined for each listing from the first set of search listings wherein the confidence score is determined in accordance with a relevance of each listing when compared to the listings of the second set of search listings. The identified search listings from the first set of search listing are ordered in accordance, at least in part, with the confidence score for each search listing.